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Autonomous  Quality  Space  Imagery  For 
LEO/GEO  Space  Operations 


David  K.  Geller*  John  Mclnroyj 

Utah  State  University ,  Logan,  Utah,  84322 


Quality  space  imagery  is  required  for  many  new  and  innovative  LEO/GEO 
missions  including  satellite  inspection,  servicing,  and  docking,  as  well  for 
general  space  situational  awareness.  Current  space  imagery  capabilities  re¬ 
quire  “experts”  to  be  on-site  at  a  mission  operation  center  to  conduct  image 
data  analysis,  mission  re-planning,  system  analysis,  and  space  vehicle  com¬ 
manding.  This  requires  a  significant  amount  time  and  money  as  well  as 
additional  delays  waiting  for  downlink/uplink  opportunities.  The  goal  of 
this  paper  is  to  present  strategies  and  techniques  that  will  provide  a  more 
autonomous  approach  to  collecting  quality  space  imagery.  This  includes 
autonomous  image  analysis,  mission  planning  capabilities,  and  GNfcC  al¬ 
gorithms  that  can  be  implemented  and  executed  onboard  the  space  vehi¬ 
cle.  The  implementation  of  these  strategies  and  techniques  will  reduce  the 
amount  of  time  and  effort  required  of  the  mission  operations  centers,  reduce 
dependence  on  downlink/uplink  opportunities,  and  provide  space  vehicles 
that  can  be  more  responsive  to  customer  input. 

I.  Introduction 

The  long  term  goal  of  this  effort  is  to  move  quality  space  imagery  activities  from  ground- 
based  mission  operations  centers  onto  the  space  vehicle  to  create  a  more  autonomous  ca¬ 
pability.  Two  scenarios  that  cover  a  wide  spectrum  of  applications  are  being  considered. 
The  first  is  an  autonomous  survey  of  an  unknown  or  partially  known  space  object.  For  this 
scenario,  an  optimal  trajectory  plan  would  be  developed  onboard  the  space  vehicle  such  that 
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a  survey  of  the  object  could  be  conducted  within  desired  customer  time  constraints.  During 
the  survey,  basic  image  metrics  such  as  lighting,  under/over-exposure,  spatial  resolution, 
and  contrast  would  be  evaluated.  Trajectory  modifications  would  be  made  if  required,  and 
images  would  be  saved  and  stored  for  downlink  to  the  ground.  The  second  scenario  is  for 
autonomous  health  monitoring  or  problem  evaluation  and  diagnosis  for  a  well  known  object. 
In  this  scenario  the  objective  is  to  obtain  very  precise  and  detailed  imagery  of  the  object  or 
a  component  of  the  object.  Detailed  image  analysis  is  conducted  onboard  the  space  vehicle 
and  metrics  for  lighting,  contrast,  glare,  exposure,  blurring,  resolution,  etc,  are  computed. 
The  metrics  are  then  used  to  adjust  camera  settings  and  to  compute  a  desired  space  vehicle 
position  and  orientation.  Onboard  mission  managers  and  trajectory  planners  then  determine 
an  optimal  and  safe  sequence  of  translation  and  rotational  maneuvers,  and  an  autonomous 
onboard  GN&C  system  can  navigate  and  execute  the  maneuvers  to  achieve  the  desired  po¬ 
sition  and  orientation.  Both  of  these  scenarios  can  be  captured  in  the  ideal  autonomous 
quality  space  imagery  system  shown  in  the  conceptual  block  diagram  below. 


Desired  Quality  Imagery  Quality  Imagery 
(from  customer)  (to  customer) 


Figure  1.  An  Ideal  autonomous  quality  space  imagery  system 

The  basic  components  of  this  system  are:  1)  the  space  vehicle  inspection  camera  and 
its  image  processing  and  analysis  algorithms  needed  to  assess  the  quality  of  an  image,  2) 
the  environment  and  image  device  parameter  selection  algorithms  that  will  improve  the 
image  quality,  3)  the  autonomous  mission  and  trajectory  planning  algorithms  to  determine 
optimal  and  safe  maneuver  sequences,  and  4)  the  GN&C  algorithms  required  to  implement 
the  trajectory  commands  and  mission  plan.  These  four  components  are  discussed  in  more 
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detail  below. 


II.  Image  Processing,  Analysis,  and  Image  Quality  Metrics 

This  component  of  the  system  contains  the  image  processing  algorithms  that  compute  the 
image  quality  metrics  needed  to  assess  the  quality  of  a  space  image.  These  metrics  include 
image  contrast,  blurring,  signal-to- noise,  lighting,  and  resolution.  The  following  metrics  are 
also  of  crucial  important  for  autonomously  obtaining  high  quality  images:  detection  of  under 
exposure,  over  exposure,  and  excessive  glare.  These  would  be  identified  by  noting:  (1)  A 
loss  in  texture  within  the  target;  (2)  a  very  bright  image;  or  (3)  a  very  dark  image.  The 
algorithms  used  to  compute  these  metrics  are  varied.  In  this  paper,  we  focus  on  extracting 
the  lighting  and  resolution  metrics  from  the  images.  The  algorithms  employed  to  do  this  are 
described  below. 

A.  Spatial  Resolution 

The  spatial  resolution  of  the  target’s  image  determines  the  smallest  features  that  can  be 
distinguished,  and  it  is  influenced  principally  by  the  range  and  the  zoom.  Spatial  resolution 
is  especially  fruitful  to  study  because  it  is  not  commonly  automated  in  commercial  off-the- 
shelf  (COTS)  camera  systems,  yet  it  is  a  key  factor  in  image  quality.  It  is  not  automated 
because  it  requires  understanding  what  you  are  viewing.  Normally,  a  human  is  looking  in 
the  camera  (in  the  loop)  to  adjust  the  zoom  and  move  closer  if  necessary.  This  is,  of  course, 
not  the  case  when  space  images  are  captured  autonomously,  so  the  need  for  automation  is 
great.  In  contrast,  other  imaging  parameters  such  as  focusing  (blurring)  or  exposure  settings 
already  have  been  automated  in  COTS  cameras,  so  much  less  effort  needs  to  be  given  to 
them.  The  following  steps  determine  the  current  spatial  resolution. 

1.  The  portion  of  the  image  corresponding  to  the  target  satellite  is  segmented  from  the 
rest  of  the  image,  and  its  centroid  is  found. 

2.  This  information  is  then  used  to  determine  a  fine  pointing  vector  between  the  target 
and  chaser.  (It  is  assumed  that  a  coarse  pointing  vector  is  already  available,  i.e.  that 
initial  acquisition  has  already  been  completed.) 

3.  From  the  current  parameters  (range,  aperture  size,  zoom),  the  spatial  resolution  of  the 
target’s  image  is  determined. 
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To  maintain  a  constant  resolution  despite  changes  in  the  distance  to  the  target,  the 
camera’s  view  angle,  0view ,  (in  degrees)  is  set  according  to: 


_  sizepixnumpix  180 

u view 

r  7 r 


(i) 


where  sizepix  is  the  size  of  a  pixel  (in  meters),  is  the  number  of  pixels  across  the 

image,  and  r  is  the  range,  or  distance,  between  the  chaser  and  target  (in  meters).  This  paper 
neglects  diffraction  effects,  although  they  will  be  included  in  our  future  work. 


B.  Lighting  Vector 

Space  images  often  contain  extreme  glare  and  dark  shadows,  but  with  minor  adjustments, 
major  improvements  to  the  lighting  can  be  obtained.  This  is  achieved  by  repositioning  the 
chaser  s/c.  The  direction  to  move  the  chaser  is  determined  by  the  lighting  vector.  Let  ca 
denote  the  centroid  of  a  binary  image.  It  is  given  by 


Co, 


-E 

n2j 


(2) 


where  N  is  the  total  number  of  pixels  in  the  object’s  image,  O  is  the  set  of  all  object  pixels, 
j  is  the  x  location  (column)  of  an  object  pixel,  and  i  is  the  y  location  (row)  of  an  object 
pixel. 

The  center  of  intensity  is  given  by 


Ci  = 


1 

It 


£7(U) 


(3) 


where  I(i,j )  is  the  intensity  of  the  pixel  in  row  i  and  column  j,  and  It  is  the  total  intensity 

It  =  EoFb /)• 

The  lighting  vector  is  then  given  by 


vL  =  Ci-  ca  (4) 

Translating  the  chaser  s/c  in  the  direction  of  the  lighting  vector  positions  the  chaser  so  it 
can  see  the  illuminated  side  of  the  target  s/c.  However,  the  lighting  vector  is  in  image 
coordinates,  so  they  must  be  converted  to  the  chaser’s  local  coordinate  system.  Let  Rc, image 
be  the  rotation  matrix  from  the  image  coordinates  to  the  chaser  coordinates.  The  direction 
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of  the  lighting  vector,  in  chaser  coordinates,  is  then 


(5) 


The  chaser  s/c  is  then  translated  by 


Apc  =  g 


IKc 


(6) 


where  g  is  a  control  system  gain. 

III.  Environment  and  Image  Device  Parameters  Selection 

The  environment  and  image  device  parameters  include  sun  and  moon  angles,  lunar  phase 
angle,  earth  horizon  angle,  time-to-eclipse  entrance/exit,  range-to-target,  relative  orienta¬ 
tion,  camera  exposure  setting,  focus,  and  time  of  image.  The  current  analysis  is  focused 
on  algorithms  that  will  select  sun-angle,  range,  orientation,  and  zoom  to  obtain  a  desired 
resolution  and  lighting  condition.  For  this  analysis  the  sun-angle  is  simply  set  at  noon,  and 
the  orientation  of  the  space  vehicle  is  constrained  to  point  at  the  object  with  an  arbitrary 
roll  angle  about  the  camera  bore  sight.  New  zoom  and  range  set  points  that  result  in  an 
image  meeting  the  spatial  resolution  specification  are  found  using  (1)  above. 

IV.  Autonomous  Mission  and  Trajectory  Planning 

The  mission  planner  will  be  required  to  autonomously  plan  the  activities  of  of  the  space¬ 
craft  for  as  little  2-3  orbits  and  up  to  2-3  days  of  activities.  A  typical  mission  includes  an 
Initial  Acquisition  Phase  to  locate  the  object  of  interest,  and  Approach  Phase  to  safely  get  in 
the  proximity  of  the  object,  a  Parking  Phase  for  vehicle  health  monitoring  and  orbit  phasing, 
a  Survey  Phase  for  collecting  detailed  imagery  in  the  proximity  of  the  object,  and  a  Mission 
Begin/End  phase  to  plan  uplink/downlink  activities.  An  additional  Active  or  Passive  Abort 
Phase  is  used  when  anomalies  are  observed  or  detected. 

The  mission  and  trajectory  planner  will  also  have  key  mission,  trajectory,  and  space¬ 
craft  parameters  and  constraints  that  govern  and/or  constrain  the  maneuvering  capabilities 
(translation  and  rotational)  of  the  space  vehicle.  These  include  maximum  rotational  rates, 
maximum  translational  acceleration,  orientation  constraints  (for  sensor  operations,  commu¬ 
nications,  and  solar  power),  closest  approach  distances  and  associated  relative  velocities,  fuel 
levels  and  acceptable  maneuver  An  levels,  mission  time  limitations,  and  power  constraints. 

The  current  analysis  has  focused  on  two  components  of  mission  and  trajectory  planning: 
1)  the  calculation  of  An  required  to  station- keep  at  a  desired  sun- angle  and  range  or  the 
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Av  required  for  inertial  station-keeping,  and  2)  the  calculation  of  optimal  maneuvers  for 
proximity  operations. 

The  Av  required  to  station-keep  at  a  desired  range  and  sun-angle  can  be  derived  from 
the  CW  equations 

fLVLH  =  ft LVLH  +  ARLVLH  +  BRLVLH  (7) 

where  HLVLH  and  its  time  derivatives  describe  the  relative  motion  of  the  spacecraft  in  a 
local-vertical  local-horizontal(LVLH)  frame,  and  fLVLH  are  the  non-gravitational  specific 
forces.  The  matrices  A  and  B  are  given  by 


0 

0 

0 

0 

0 

‘2.Uj0rb 

0 

k' orb 

0 

,  B  = 

0 

0 

0 

_0 

0 

-3  ^orb. 

2  (jJorb 

0 

0 

(8) 


where  uoorb  is  the  orbital  frequency.  The  origin  of  the  LVLH  frame  is  located  at  the  center 
of  mass  of  the  target  vehicle,  and  the  components  of  ~R}vlh  are  altitude,  downrange,  and 
crosstrack.  The  above  CW  equations  are  valid  when  the  vehicles  are  in  near-circular  orbits 
and  relatively  close  to  one  another. 

As  an  inspection  spacecraft  maintain  constant  range  and  sun  angles  relative  to  a  space 
object  (i.e.  inertial  station- keeping) ,  the  relative  motion  in  the  LVLH  frame  traces  out  a  circle 
as  shown  in  Fig  ??.  This  relative  motion  trajectory  is  generated  by  rotating  the  position 
vector  of  the  inspector  about  the  orbit  normal  at  the  orbital  frequency  (in  the  opposite 
direction).  This  is  easily  described  by 


KLVLH(At)  =  R 


sin(0)  cos(u>orbAt  +  xji) 
cos  ((f)) 

sin(0)  sin(a;0).ftAt  +  xji) 


(9) 


where  R  is  the  relative  range  between  the  craft,  <fi  is  the  angle  between  RLVLH  and  the  orbit 
normal,  and  ip  is  a  phase  angle  defining  the  location  of  the  vehicle  at  the  initial  time  tz. 

The  Av  required  for  station-keeping  under  constant  thrust  for  a  time  At  can  now  be 
found  by  substituting  (9)  and  its  associated  time  derivatives  into  (7).  This  results  in  an 
expression  for  the  specific  force  required  to  maintain  constant  range  and  constant  sun-angles 
relative  to  the  space  object  (i.e  to  maintain  a  constant  inertial  position  relative  to  the  space 
object) 

sin (cj))  cos (lv orb  At  +  ip) 

COS  ((f)) 


(10) 


{(At)  =  II. 4;.,, 


—2  sin(0)  sin  (a;^  At  +  tp) 
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The  integration  of  (10)  over  time  results  in  an  expression  for  the  An  required  for  inertial 
station-keeping  over  a  time  At. 


rAt  R  (r  / - 

Avsk  =  /  |f  (r)|  dr  =  47T2—  /  yl  +  3  sin2(0)  sin2(27m  +  ip)dn  (11) 

Jo  P  Jo  v 

where  P  is  the  orbital  period. 

In  an  effort  to  characterize  this  equation  three  cases  are  considered  over  the  course  of 
two  orbits.  For  each  case  the  <p  is  varied  from  0°  to  90°  in  10°  increments.  These  results  are 
shown  in  Fig.  2.  Noting  that  the  term  contains  all  the  dimensionality  in  (11),  allows  the 


DELTA  V  used  to  Station-Keep 


Figure  2.  Examples  of  the  Av  used  for  station- keeping  as  a  function  of  time  for  LEO,  MEO, 
and  GEO. 


reformulation  of  (11) 


where 


A  R 

Av  =  - 


K(4>,n)-K(4>, 


ip 


P  iOorb 


R  fn  / - 

K(cf),n)  =  47t2—  /  y  1  +  3sin2((/>)  sin2(27 rn)dn 

P  J  o  v 


(12) 


(13) 


Values  for  K(<p,n)  have  been  computed  and  are  found  in  Fig  3.  Based  on  the  data  in  the 
above  figure,  autonomous  onboard  estimates  of  inertial  station-keeping  An  can  be  easily 
accomplished. 

It  will  also  be  important  to  minimize  the  propellant  used  during  proximity  operations.  If 
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Nondimentional  Abused  to  Station-Keep 


Figure  3.  Examples  of  the  Av  used  for  station-keeping  as  a  function  of  time. 


constant  lighting  conditions  are  achieved  by  using  inertial  station-keeping,  the  CW  equations 
can  be  used  to  determine  an  optimal  transfer  time  from  the  current  range  and  lighting 
conditions  to  any  new  range  and  lighting  condition. 

The  basic  approach  is  to  use  the  CW  equations  expressed  in  matrix  form. 


R  lvlh(t0  +  At) 

$rr(A  t) 

$n>(A  t) 

Rlvlh(t0) 

\lvlh(t0  +  At) 

^vr(At) 

t) 

Vlvlh(to)  +  Av 

(14) 


where  At  is  the  time  between  maneuvers,  Av  is  change  in  velocity  due  to  the  maneuvers, 
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and  the  transition  matrices  are  given  by 


^rr(At)  = 


<S>rv(At)  = 


®vr(At)  = 


$vv{A  t)  = 


4  —  3  cos(uorbAt)  0  0 

6sin(ujorbAt)  —  6ujorbAt  1  0  (15) 

0  0  cos(u>orbAt ) 

sin(a;or6  At)  /  u>orb  2{T  cos((n07.^At)}‘//(n07.fr  0 

2{cos(u)orbAt)  —  1} / LUorb  4  sm(uorbAt)/uorb  —  3At  0  (|16) 

0  0  sm(u>orbAt)/u>orb 

3cu  orb  sm(uj  orb  At)  0  0 

6c<;or(){cos(a;orfcAt)  —  1}  0  0  (17) 

0  0  —  ujorb  sin(avfoAt) 

cos(uorbAt )  2  sin(a;orfcAt)  0 

— 2  sin(a;orfcAt)  4cos(a;orf>A£)  —  3  0  (18) 

0  0  cos(cjorbAt) 


Since  it  is  assumed  that  the  vehicle  will  be  station  keeping  before  the  transfer,  the  velocity 
prior  to  the  transfer  is  given  by 


RfVLn  =  rlvlh  x  w  (19) 

where  c o  is  directed  along  the  orbit  normal  and  has  a  magnitude  of  ojoti,  . 

A  fixed  target  position  in  the  inertial  frame  is  equivalent  to  a  time- varying  target  position 
in  the  LVLH  frame  that  can  be  described  by  (9).  Using  the  final/target  position  in  the 
LVLH  as  a  function  of  the  transfer  time,  the  Av  corresponding  to  the  first  maneuver  can  be 
determined  as  function  of  the  transfer  time.  This  is  accomplished  by  solving  the  CW  for  the 
needed  velocity  and  then  subtracting  the  initial  velocity. 

Avi(At)  =  $-x(A t)  (RfVLH (At)  -  $rr(At)R*:VLH)  -  R^VLH  x  u  (20) 

Since  it  is  assumed  that  the  inspector  will  be  required  station-keep  after  the  transfer,  the 
desired  final  velocity  can  also  be  written  as  a  function  of  the  transfer  time. 

Vf(At)  =  Rf(At)LVLH  x  u>  (21) 

The  Av  corresponding  to  the  second  maneuver  can  be  determined  by  again  using  the  CW 
equations  to  find  the  velocity  at  the  end  of  the  transfer.  This  is  subtracted  from  the  de¬ 
sired  final  velocity.  Substituting  the  sum  of  (19)  and  (20)  for  the  initial  velocity  yields  an 
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expression  for  the  Av  for  the  second  maneuver. 


Av2(At)  =Rf(At)LVLH  xu>-  <S>vrRt;VLH 

-  $W(A t)  (RjVLH(A t)  -  $rr(At)R™)]  (22) 

The  optimal  transfer  time  can  be  found  by  minimizing  the  performance  index  J,  the  sum  of 
the  magnitudes  of  the  two  maneuvers. 

J(At)  =  |Avi(At)|  +  |Av2(Af)|  (23) 

A  sample  case  showing  the  effect  of  transfer  time  in  total  Av  is  shown  in  Fig.  4.  These  types 


AV  cost  as  a  function  of  At 


Figure  4.  Example  of  transfer  from  a  sun  angle  of  20°  and  a  range  of  175m  to  a  sun  angle  of 
45°  and  a  range  of  50m  in  LEO. 

of  optimization  problems  are  not  difficult  to  solve  and  can  easily  be  added  to  an  onboard 
GN&C  system. 


V.  GN&C  Algorithms 

A  package  of  simulation  models  and  basic  GN&C  algorithms  has  been  developed  for 
this  project.  This  package  includes  6-DOF  dynamics  for  each  spacecraft;  star-trackers,  gy- 
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ros,  camera,  accelerometers,  an  RCS  thruster  configuration,  and  momentum  wheels  for  the 
inspector  spacecraft;  maneuver  targeting  algorithms,  attitude  determination  and  control 
algorithms,  jet-select  algorithms,  and  station-keeping  algorithms  for  the  onboard  GN&C 
system. 


VI.  Current  Results 


A.  Lighting  Improvement 

Figure  5  illustrates  a  typical  (simulated)  space  image  consisting  largely  of  glare  and  dark 
shadows.  For  this  image,  the  centroid,  ca,  is  computed  using  (2)  and  drawn  as  a  blue  *. 
Additionally,  the  center  of  intensity,  q,  is  computed  and  drawn  as  a  green  pentagon.  The 
lighting  vector  vl  =  q  —  ca  is  computed  using  (4)  and  (5).  Finally,  a  translational  command, 
A pc,  is  calculated  using  (6).  The  resulting  image,  with  improved  lighting,  is  illustrated  in 
Fig.  6.  Note  that  the  glare  and  shadows  have  been  significantly  reduced. 

B.  Spatial  Resolution 

Simulations  have  also  been  performed  which  automatically  adjust  the  zoom  to  maintain  a 
constant  spatial  resolution  as  the  target  range  changes.  Figure  7  illustrates  a  distant  view  of 
the  target,  which  has  occurred  as  a  result  of  optimal  chaser  maneuvers.  Note  that  the  target 
image  occupies  a  small  image  area,  therefore  the  spatial  resolution  is  small.  To  prevent  this 
loss  of  spatial  resolution,  the  camera’s  view  angle  is  adjusted  using  equation  (1).  Figure  8 
illustrates  the  same  scene  when  the  zoom  is  automatically  adjusted.  Note  that  the  target 
now  fills  the  image.  Note  also  that,  because  the  camera  view  angle  is  now  much  smaller, 
only  a  few  stars  remain  in  the  image. 

To  date,  only  the  zoom  has  been  adjusted.  However,  future  versions  are  anticipated  which 
will  adjust  both  the  chaser  position  and  the  zoom,  so  spatial  resolution  can  be  maintained 
despite  diffraction  effects. 
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Figure  5.  An  Image  with  Poor  Lighting. 
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Figure  6.  Autonomously  Maneuvering  the  Chaser  Slightly  to  the  Right  Dramatically  Improves 
the  Lighting. 
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Figure  7.  A  Distant  View. 
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Figure  8.  The  Distant  View,  When  Zoom  is  Autonomously  Adjusted  to  Maintain  Spatial 
Resolution. 
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VII.  Conclusions 


In  order  to  autonomously  obtain  quality  imagery  of  space  objects,  a  collection  of  onboard 
mission  and  trajectory  planners,  GN&C  algorithms,  and  image  processing  algorithms  must 
be  brought  together  and  utilized  in  a  closed-loop  fashion  to  produce  images  that  meet  re¬ 
quired  metrics  as  defined  by  the  customer.  This  paper  has  taken  the  first  step  in  identifying 
the  procedures  and  algorithms  that  are  required  to  accomplish  this  goal.  Several  of  these 
procedures  and  algorithms  have  been  developed  and  implemented  in  closed-loop  simulation 
with  good  preliminary  results.  Some  of  the  procedures  and  algorithms  discussed  thus  far 
include:  1)  extraction  of  lighting  conditions  from  images,  2)  calculation  of  image  resolution, 
2)  calculation  of  desired  position  and  orientation  to  achieve  better  lighting  conditions,  3)  es¬ 
timation  of  station-keeping  maneuver  An,  and  4)  optimization  of  maneuver  Av  for  proximity 
operations. 
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Appendix  A:  If  Needed 
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